Taylor Series Expansions for Stationary Markov Chains

Series Expansions for Finite-state Markov Chains

This paper provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

Taylor Expansion for the Entropy Rate of Hidden Markov Chains

We study the entropy rate of a hidden Markov process, defined by observing the output of a symmetric channel whose input is a first order Markov process. Although this definition is very simple, obtaining the exact amount of entropy rate in calculation is an open problem. We introduce some probability matrices based on Markov chain's and channel's parameters. Then, we try to obtain an estimate ...

4 Singular Perturbations of Markov Chains and Decision Processes

In this survey we present a uni ed treatment of both singular and regular perturbations in nite Markov chains and decision processes. The treatment is based on the analysis of series expansions of various important entities such as the perturbed stationary distribution matrix, the deviation matrix, the mean-passage times matrix and others. 4.1 BACKGROUND AND MOTIVATION Finite state Markov Chain...

Nonlinearly Perturbed Stochastic Processes and Systems

This paper is a survey of results presented in the recent book Gyllenberg and Silvestrov [GS08]. This book is devoted to studies of quasi-stationary phenomena for nonlinearly perturbed stochastic processes and systems. New methods of asymptotic analysis for nonlinearly perturbed stochastic processes based on asymptotic expansions for perturbed renewal equation and recurrence algorithms for cons...

Sieve bootstrap with variable length Markov chains for stationary categorical time series